The present invention relates to a moving picture synthesizing device for synthesizing plural moving picture code strings, and obtaining a moving picture code string describing a synthesized moving picture composed by including moving pictures described by these plural moving picture-code strings.
A conventional moving picture synthesizing device for synthesizing plural moving picture code strings, and obtaining a moving picture code string describing a synthesized moving picture composed by including moving pictures described by these plural moving picture code strings is described below.
FIG. 36 is a block diagram of an example of a conventional moving picture synthesizing device. This conventional device decodes coded strings 1, 2, 3, 4 of moving pictures into picture data 6, 7,8, 9 for one frame by a decoder 5, synthesizes into synthesized picture data 11 by a picture synthesizer 10. A coder 12 codes the synthesized picture data 11 with frame by frame, and generates a synthesized moving picture code string 13. This conventional device requires a tremendous amount of computation for decoding all moving picture code strings of the materials to be synthesized, synthesizing picture, and coding again the picture data of synthesized result.
Japanese Laid-open Patent Publication No. 7-298263 discloses a device for dividing an input moving picture into plural regions on a picture, synthesizing moving picture code strings obtained by coding in each divided region, and generating a moving picture code string describing the original moving picture. FIG. 37 is an explanatory diagram showing the synthesizing action for synthesized moving picture code string described original moving picture, disclosed in Japanese Laid-open Patent Publication No. 7-298263.
First as shown in FIG. 37a, this device divides continuous frames 1, 14 into three sections in the vertical direction in the picture region, and generates code strings as shown in FIG. 37b in each divided region. These code strings are linked on the boundary of frames and between frames in each slice. That is, in frame 1, code strings 5a, 6a,7a,8a are linked in this sequence, and a code string as shown in FIG. 37c is generated. The linked code string has the same syntax as in the case of coding without dividing the picture.
In this device, each frame picture of the original moving picture is divided into predetermined regions, and coded in each divided region, At this stage of coding, already, the linking sequence of code strings in each divided region is fixed. According to this fixed procedure, the code strings in each divided region are linked, and a code string for the entire original moving picture is generated. Therefore, it is not allowed to select plural arbitrary moving picture code strings, and obtain a code string corresponding to a synthesized moving picture composed by disposing the moving pictures described by these moving picture code strings at arbitrary positions.
Further, Japanese Laid-open Patent Publication No. 10-210465 discloses a device for generating a moving picture code string for describing a synthesized moving picture composed by including the moving pictures described by each code string, from plural moving picture code strings. This device changes over use or non-use of a picture header included in each one of plural moving picture code strings, and replaces or deletes a slice header included in each one of the plural moving picture code strings. And this device links the plural code strings into a synthesized moving picture code string.
This device does not comprise any means for generating various header information for entire synthesized moving picture. So, this device does not regenerate a information about picture size, picture rate, aspect ratio, bit rate, video buffering verifier buffer size, motion vector processing precision, motion vector description range, picture coding type, and others. And hence the obtained xe2x80x9cmoving picture code string for describing the synthesized moving picturexe2x80x9d may not be decoded correctly.
The moving picture synthesizing device of the invention is to synthesize plural moving picture code strings, and obtain a moving picture code string for describing a synthesized moving picture what includes moving pictures described by these plural moving picture code strings disposed in an arbitrary configuration on a picture. In particular, it is intended to describe the synthesized moving picture correctly by the picture headers and slice headers included in the obtain ed synthesized moving picture code string.
To realize this, the moving picture synthesizing device of the invention comprises code string analyzer for analyzing the header information of moving picture code strings used for synthesis, picture information generator for generating header information of synthesized moving picture code string for describing the synthesized moving picture to be generated by using the analyzed result, and code string rewriting part for rewriting the display position information described in the moving picture code strings used for synthesis. Adequate header information corresponding to the synthesized moving picture is generated by a simple computation, and the moving picture code strings for describing the synthesized moving picture efficiently is generated without decoding the moving picture code strings used for synthesis.
In other aspect of the invention, the moving picture synthesizing device can process easily and at high speed when obtaining synthesized moving picture code string describing the synthesized moving picture and consisting of a mixture of different frame types such as intra-frame and predictive frame, without deteriorating the picture quality of the original moving picture, without complicating the constitution of the computer, or without using expensive or high performance device.
To realize this, the moving picture synthesizing device in this aspect of the invention includes picture coder. The picture coder generates and issues moving picture code strings for synthesis consisting of a mixture of multiple frame types such as intra-frame and predictive frame when compressing and coding input video signal. At this time, the picture coder processes all moving picture code strings for synthesis to be issued so that the occurrence pattern of frame types may be always the same. This device reads in the input of plural moving picture code strings for synthesis so that the occurrence pattern of frame types may be all the same, synthesizes in the frame unit, and issues the synthesized moving picture code string similarly in the same pattern.
The picture coder generates a GOP (group of pictures), and processes at this time so that the occurrence pattern may be same in each GOP, concerning the frame types of each frame in the GOP, and that the number of frames included in the GOP may be a fixed number. As a result, by sorting the moving picture code strings for synthesis at random in GOP unit, and the picture can be synthesized by using the moving picture code strings after sorting.
In a different aspect of the invention, the moving picture synthesizing device processes the input of video signal by decimating between frames in a specific frame period, feeds the decimated video signal into picture coder. And, this device generates and inserts frame picture code strings describing the same frame picture as either preceding or succeeding frame picture, or an intermediate frame picture of preceding and succeeding frames, into synthesized moving picture code string what is synthesized from the moving picture code strings coded after decimation, so that a playing speed of moving picture described by the inserted picture code string may be equal to the time axis playing speed of the video signal before decimation. As a result, the number of frames per unit time is smaller than in the original input video signal, and the picture coding process and picture synthesizing process per unit time may be curtailed, and the bit rate can be lowered. Furthermore, the storage capacity of the moving picture code strings for synthesis may be reduced when storing in storage.
In a further aspect of the invention, the moving picture synthesizing device includes video division setting part for setting video division frame positions, by detecting the division frame positions as the divisions of input video signal either automatically or manually, and the frames included in the video division are adjusted by inserting dummy frames or decimating frames, so that the number of frames in the video division may be a multiple of a fixed number of frames included in the GOP, thereby synthesizing randomly in the video division unit.
In other different aspect of the invention, the moving picture synthesizing device includes frame selector for picking up frame-picture code strings having a specific frame type, from the GOP of each moving picture string for synthesis to be synthesized, and the synthesized frame code string is generated only from the frame code strings of the specific frame type.
It hence requires only the process of synthesizing the frame code strings belonging to the specific frame type (for example, intra-frames only), so that the load of synthesizing process is alleviated, while the efficiency is enhanced.
In a further different aspect of the invention, the moving picture synthesizing device has the picture coder which is constituted so as to divide, compress, and code a picture of the divided region in the frame picture of the input video signal, in the unit of DCT block, macroblock, or GOB (group of blocks), and, within the moving picture code strings for synthesis to be synthesized, the frame data of intra-frames are converted into frame data of predictive frames in the unit of the picture regions. At this time, when compressing and coding the data in the divided region by the picture coder, the type of motion prediction (forward prediction, backward prediction, etc.) and block type such as intra code or not are given and issued as attributes in each picure region. At the time of synthesis, each block type of the moving picture code strings for synthesis is judged, and properly converted or processed.
As a result, even from any arbitrary frame position other than the GOP unit, it is possible to synthesize by minimizing the picture quality deterioration by decoding. Moreover, when converting the frame type, conversion can be controlled in the block unit, and useless conversion process can be avoided. It is also possible to synthesize in the block unit. Still more, since the individual frame types can be converted, if the frame patterns of the video signal being taken in are random, it is possible to obtain synthesized picture code strings flexibly without being restricted.
In another different aspect of the invention, the moving picture synthesizing device generates and inserts frame picture code strings describing the same frame picture as either preceding or succeeding frame picture, or an intermediate frame picture of preceding and succeeding frames, individually, into moving picture code strings for synthesis. As a result, in each picture region of the frame picture of the synthesized moving picture describing the synthesized moving picture code strings, it is possible to generate synthesized moving picture code strings describing the synthesized pictures presented at different reproduction speeds, in the portion corresponding to the moving picture code strings for synthesis in which the frame picture code string is inserted, and in the portion corresponding to the moving picture code strings for synthesis in which nothing is inserted.